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FOREWORD 


This  experiment  was  conducted  in  support  of  advanced  development  project  Z1177 
(Advanced  Computer-aided  Instruction),  subproject  Z 1177-004  (Remote  Site  Training 
Using  Microcomputers)  and  was  sponsored  by  the  Chief  of  Naval  Operations  (OP -01).  The 
purpose  of  this  subproject  was  to  conduct  training  research  to  improve  electronic  counter¬ 
countermeasures  (ECCM)  training  readiness  in  the  surface  fleets. 

This  report  presents  an  experiment,  conducted  in  conjunction  with  the  subproject, 
that  investigated  the  overall  effectiveness  of  using  computer  graphics  for  teaching  Navy 
personnel  a  visual  recognition  skill  (specifically,  recognizing  and  identifying  radar 
jamming).  The  value  of  animation  (motion)  and  a  brief  verbal  description  was  also 
explored.  The  results  are  intended  for  the  use  of  members  of  the  military  training 
community  who  are  involved  with  training  development,  computer-based  graphics, 
simulation,  and  visual  recognition  training. 

The  outstanding  cooperation  of  personnel  at  the  Fleet  Combat  Training  Center, 
Pacific  is  very  much  appreciated. 


B.  E.  BACON 
Captain,  U.S.  Navy 
Commanding  Officer 


3.  S.  McMICHAEL 
Technical  Director 
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SUMMARY 


Problem  and  Background 

Interest  in  computer  graphics  has  increased  with  the  advent  of  computer-based 
instruction.  However,  since  computer  graphics  increase  instructional  development  costs 
significantly,  we  need  to  know  whether  they  are  necessary  and,  if  so,  how  they  should  be 
used.  Probably,  the  importance  of  computer  graphics  depends  largely  on  the  task  to  be 
learned. 

This  experiment  was  conducted  in  conjunction  with  a  computer-based  course  teaching 
a  visual  recognition  skill;  that  is,  recognition  of  radar  jamming.  This  course  teaches 
operations  specialists  and  fire  control,  electronics,  and  electronics  warfare  technicians  to 
recognize  11  types  of  radar  jamming.  Actual  jamming  can  be  difficult  to  identify  because 
the  patterns  are  complex  and  their  critical  features  ambiguous.  Therefore,  computer 
graphics  were  developed  to  enhance,  clarify,  and  exaggerate  the  unique  or  distinguishing 
features  of  each  type  of  jamming. 

Purpose 


This  experiment  was  conducted  to  investigate  the  overall  effectiveness  of  using 
computer  graphics  for  teaching  Navy  personnel  a  visual  recognition  skill;  specifically, 
recognizing  and  identifying  radar  jamming.  The  value  of  animation  (motion)  and  a  brief 
description  was  also  explored. 

Method 

The  experimental  subjects  received  some  preview  instruction  before  they  took  the 
computer-based  jamming  recognition  course.  Their  performance  on  the  pretest  for  this 
course  was  compared  with  that  of  a  control  group  that  received  no  preview  instruction. 

The  experimental  subjects  were  114  male  enlisted  personnel  attending  electronic 
counter-countermeasure  and  advanced  warfare  classes  at  the  Fleet  Combat  Training 
Center,  Pacific.  The  control  group  was  formed  from  data  previously  collected  from  93 
students  who  had  attended  the  computer-based  course. 

The  experimental  subjects  were  randomly  divided  into  five  groups  according  to  which 
of  the  following  preview  conditions  they  received:  static  (nonmoving)  graphics  with  and 
without  descriptions,  dynamic  (moving)  graphics  with  and  without  descriptions,  and 
descriptions  only.  The  students  took  the  preview  training  on  one  day  and,  after  a  one-day 
delay,  took  the  pretest  of  the  computer-baseH  course.  Mean  pretest  scores,  response 
times,  and  total  preview  times  of  all  groups  were  compared. 

Results  and  Discussion 


1.  The  groups  using  the  dynamic  graphics  (with  and  without  descriptions)  scored 
statistically  higher  on  the  pretest  than  did  the  other  groups  (including  the  control  group, 
which  received  no  preview  training). 

2.  The  differences  among  the  mean  response  times  of  the  experimental  and  control 
groups  were  not  statistically  significant. 


3.  The  mean  total  time  that  the  subject  groups  spent  on  the  preview  condition 
differed  significantly  among  the  groups.  The  dynamic-graphics-with-descriptions  group 
spent  more  time  viewing  the  graphics  than  did  the  other  groups. 

Our  findings  suggest  that  considering  the  prototypical  features  or  characteristics  of 
the  visual  patterns  to  be  learned  is  important  in  developing  the  graphics  and  that  the  use 
of  descriptions  added  to  the  value  of  the  dynamic  graphics. 

Conclusions 

Dynamic  graphics  that  consider  the  characteristics  of  the  visual  patterns  to  be 
learned  appear  to  be  a  useful  way  of  giving  students  advance  information  about  the  visual 
characteristics  of  complex  patterns.  Descriptions  seem  to  add  to  the  value  of  dynamic 
graphics. 

Recommendation 

Training  developers  should  consider  using  dynamic  computer  graphics  that  focus  on 
the  unique  or  distinguishing  features  of  visual  patterns  to  be  learned  to  give  students 
advance  information  of  visual  characterisitcs  of  complex  patterns. 
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INTRODUCTION 


Problem  and  Purpose 

The  use  of  graphics  to  enhance  the  learning  process  has  interested  the  educational 
community  for  years.  Research  has  shown  that  the  use  of  graphics  and  text  illustrations 
can  (1)  improve  learning  by  helping  students  understand  what  they  have  read,  substituting 
pictures  for  words,  and  enhancing  student  enjoyment  and  (2)  increase  retention  of 
information  by  encoding  information  in  more  than  one  way  (i.e.,  pictures  plus  words)  and 
by  repetition  (Levie  &  Lentz,  1982). 

The  advent  of  computer-based  training  has  increased  the  interest  in  using  computer 
graphics  for  instruction  since  computer  graphics  can  be  made  to  move  and  flash  as  well  as 
be  presented  in  color.  However,  the  evidence  on  how  much  computer  graphics  contribute 
to  the  success  of  the  instruction  remains  mixed.  Since  computer  graphics  significantly 
increase  instructional  development  costs,  we  need  to  know  whether  they  are  necessary 
and,  if  so,  how  they  should  be  used  (Moore,  Nawrocki,  &  Simutis,  1979).  It  is  likely  that 
the  importance  of  graphics  depends  largely  on  the  task  to  be  learned.  For  example,  the 
computer  graphics  used  in  this  study  were  a  logical  extension  of  the  instruction  since  the 
skill  to  be  taught  was  a  visual  recognition  skill.  The  skill  was  recognition  of  radar 
jamming.  The  graphics  consisted  of  a  series  of  simplified  visual  patterns  designed  to 
organize  the  learning  of  important  visual  characteristics.  Dynamic  (moving)  graphics  as 
well  as  static  (nonmoving)  graphics  were  developed.  Description?  were  another  feature. 
Therefore,  this  experiment  explored  the  overall  effectiveness  of  using  computer  graphics 
in  training  as  well  as  the  value  of  animation  and  descriptions. 

Background 

The  present  experiment  investigates  the  influence  of  computer  graphics  on  the 
recognition  of  visual  patterns  by  Navy  personnel.  A  critical  skill  in  certain  Navy  ratings 
is  to  be  able  to  recognize  various  types  of  electronic  countermeasures  (ECMs)— also  known 
as  radar  "jamming"--that  may  obscure  targets  being  tracked  on  a  radar  scope.  Recogniz¬ 
ing  jamming  requires  differentiating  among  the  visual  characteristics  of  several  jamming 
types.  The  Navy  Personnel  Research  and  Development  Center  developed  a  successful 
computer-based  training  program  that  teaches  Navy  personnel  to  recognize  11  different 
types  of  jamming  (McDonald  &  Crawford,  1983,  1986).  This  program  combines  videotapes 
of  jamming  with  computer-based  instruction  and  graphics  that  enhance  the  visual 
characteristics  of  the  types  of  jamming. 

The  computer-based  instruction  for  recognition  of  jamming  focuses  on  categorical 
features  of  the  different  types  of  jamming.  Unique  critical  visual  features  define  and 
identify  each  type  of  jamming.  However,  actual  jamming  can  be  very  difficult  to  identify 
because  the  patterns  are  complex  and  their  critical  features  can  be  ambiguous.  For 
training,  these  features  are  difficult  to  display  in  real-life  videotape  presentations.  As  a 
result,  the  videotapes  by  themselves  were  not  sufficient  to  train  students  to  identify  each 
type  of  jamming  and  graphics  were  used  to  highlight  the  critical  features.  These  graphics 
were  "cartoon-like"  pictures  that  clarify,  enhance,  and  exaggerate  the  unique  features  of 
each  jamming  type,  much  as  political  cartoonists  exaggerate  unique  facial  features  of 
various  politicians. 

The  literature  on  concept  learning  provides  evidence  that  teaching  students  to 
recognize  a  prototypical  concept  requires  lots  of  examples  and  emphasis  on  the  prototypi¬ 
cal  features  (Gagne  &  Briggs,  1979).  The  computer  graphics  provided  examples  of  these 
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prototypical  features.  This  experiment  tested  the  use  of  computer  graphics  to  enhance 
the  critical  distinguishing  features  of  the  types  of  jamming  and  asked  the  following  three 
questions: 

1.  Are  computer  graphics  effective  in  teaching  students  to  recognize  different 
types  of  jamming? 

2.  Are  dynamic  graphics  more  effective  than  static  graphics? 

3.  Do  short  verbal  descriptions  of  the  jamming  types  enhance  the  value  of  the 
graphics? 


METHOD 

The  experiment  was  conducted  in  conjunction  with  a  course  which  taught  jamming 
recognition.  To  avoid  the  confounding  influence  of  the  jamming  recognition  instruction, 
the  experiment  was  conducted  before  the  instruction  for  the  visual  recognition  training 
course  started.  The  pretest  for  the  course,  which  presented  the  subjects  with  videotapes 
of  actual  jamming  to  identify,  served  as  the  dependent  variable.  The  experimental 
subjects  were  randomly  divided  into  five  groups  based  on  which  of  five  preview  conditions 
were  used.  The  five  preview  conditions  consisted  of  static  computer  graphics  with  and 
without  descriptions,  dynamic  computer  graphics  with  and  without  descriptions,  and 
descriptions  only.  The  performance  of  the  subjects  in  these  five  groups  on  the  pretest  to 
the  visual  recognition  training  course  was  compared  to  that  of  a  control  group  that  did  not 
receive  any  preview  instruction. 

Subjects 

The  experimental  subjects  were  114  male  enlisted  personnel  attending  electronic 
counter-counter-measure  (ECCM)  and  advanced  warfare  classes  at  Fleet  Combat  Training 
Center,  Pacific  (FCTCPAC).  Their  Navy  ratings  included  operations  specialists  (OSs),  fire 
control  technicians  (FTs),  electronics  technicians  (ETs),  and  electronics  warfare  tech¬ 
nicians  (EWs)  and  their  rates  ranged  from  E-3  to  E-7  (Table  1). 

A  control  group  of  93  subjects  was  formed  from  data  previously  collected  for 
students  who  had  attended  the  same  computer-based  course,  but  who  had  not  received  the 
preview  instruction. 

Experimental  Materials 

The  experimental  subjects  viewed  static  and  dynamic  computer  graphics  for  1 1  types 
of  jamming  in  the  following  five  preview  conditions: 

1.  Static  graphics  with  descriptions.  Two  computer  frames  each  presented  a  static 
graphic  representation  of  each  jamming  type  with  a  title  for  the  jamming  and  a  brief 
verbal  description  (approximately  one  sentence)  of  the  jamming  below  the  graphic. 

2.  Dynamic  graphics  with  descriptions.  Two  computer  frames  each  presented 
dynamic  graphic  representation  of  each  jamming  type  with  a  description  and  title  below 
the  graphic.  The  same  titles  and  descriptions  were  used  as  in  condition  1. 

3.  Static  graphics  without  descriptions.  The  same  static  graphics  were  used  as  in 
condition  1  with  only  the  jamming  title  below  each  graphic  (no  description). 
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Table  1 


Distribution  of  Subjects  by  Rating,  Rate,  and  Preview  Graphics  Group 


Rating3 
and  Rates  of 
Subjects 

Number  of  Subjects 

1 

2 

Experimental  Group*3 
3 

4 

5 

Control0 

Group 

OSSA 

1 

1 

0 

1 

1 

5 

OSSN 

3 

2 

3 

2 

3 

18 

CS3 

3 

3 

4 

3 

3 

28 

OS2 

11 

4 

5 

5 

8 

19 

OS1 

6 

7 

11 

13 

3 

17 

OSC 

0 

1 

0 

0 

0 

3 

FTM3 

0 

2 

0 

0 

0 

0 

ETN2 

0 

0 

0 

1 

0 

0 

ETR3 

0 

0 

0 

0 

1 

0 

EW2 

0 

0 

0 

0 

1 

0 

EWC 

0 

0 

0 

0 

0 

1 

FTGSN 

0 

0 

0 

0 

0 

1 

FTGCM 

0 

0 

0 

0 

0 

1 

Total 

24 

20 

25 

25 

20 

93 

3 

OS  =  operations  sepcialist 

FTM  =  fire  control  technician  (surface  missile) 

ET  =  electronics  technician 

EW  =  electronics  warfare  technician 

FTG  =  FT  (gun  fire  control) 

^The  experimental  groups  had  the  following  preview  conditions: 

Group  1.  Static  graphics  with  descriptions. 

Group  2.  Dynamic  graphics  with  descriptions. 

Group  3.  Static  graphics  without  descriptions. 

Group  4,  Dynamic  graphics  without  descriptions. 

Group  5.  Descriptions  only. 

'Control  group  received  no  preview  graphics  instructions. 
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4.  Dynamic  graphics  without  descriptions.  The  same  dynamic  graphics  were  used  as 
in  condition  2  with  only  the  jamming  title  (no  description)  below  each  graphic. 

5.  Descriptions  only.  Only  the  jamming  title  and  description  were  presented  for 
each  type  of  jamming. 

Dependent  Measures 

The  dependent  measures  in  the  experiment  were  the  students'  scores  on  the  pretest  of 
the  computer-based  ECM  recognition  training  course.  The  pretest  consisted  of  identifying 
two  videotaped  examples  of  11  types  of  actual  jamming  (22  items).  In  addition,  the 
computer  measured  subjects'  response  time  (in  seconds)  to  recognize  each  example  of 
jamming  and  the  total  time  (in  minutes)  that  each  experimental  subject  spent  in  the 
preview  condition. 

Experimental  Procedure 

Data  were  collected  from  May  1983  to  3une  1984.  The  preview  conditions  preceded 
the  regular  computer-based  ECM  recognition  training  of  ECM  and  advanced  warfare 
classes  at  FCTCPAC.  Each  of  these  classes  was  scheduled  to  be  taught  once  a  month  or 
every  two  months  and  had  8  to  10  students.  Students  from  each  class  were  assigned 
randomly  to  one  of  the  five  experimental  groups.  The  students  took  the  preview  condition 
on  one  day.  After  a  one-day  delay,  students  in  all  groups  took  the  pretest  and  proceeded 
to  the  computer-based  training  (which  is  discussed  in  McDonald  3c  Crawford,  1986). 

Equipment 

The  equipment1  used  for  this  training  included  a  TERAK  microcomputer  (an  LSI-11 
based,  dual  floppy  disk  drive  system  with  a  56K  byte  memory),  a  keyboard,  and  a  CRT 
display  for  the  presentation  of  black  and  white  graphics  and  text.  The  computer  was  used 
for  presentation  of  instruction  graphics,  test,  and  test  results,  and  for  data  collection.  A 
Betamax  videotape  player  presented  the  videotapes  of  the  jamming  on  a  TV  monitor. 


RESULTS 

Table  1  presents  the  enlisted  rates  and  ratings  of  the  subjects.  An  analysis  revealed 
no  differences  in  the  enlisted  rates  between  the  experimental  and  control  groups. 

Figure  1  shows  the  mean  pretest  scores  of  the  experimental  and  control  groups.  (The 
mean  scores  with  standard  deviations  are  listed  in  the  appendix.)  An  analysis  of  variance 
revealed  statistically  significant  differences  between  the  pretest  scores  of  groups  1 
through  4  and  those  of  groups  5  and  6  (F(l,205)  =  1.55,  p  <  .001).  Further  analysis 
revealed  that  the  groups  using  the  dynamic  graphics  (groups  2  and  4)  had  statistically 
higher  scores  on  the  pretest  than  did  the  groups  using  the  static  graphics  (groups  1  and  3) 
(F(l,92)  =  6.17,  p  <  .01).  Finally,  according  to  subsequent  tests  for  differences,  the 
dynamic-graphics-with-descriptions  group  (2)  recounted  for  the  significant  differences. 
An  analysis  of  variance  between  the  scores  oi  the  groups  with  descriptions  and  those  of 
the  groups  without  descriptions  showed  no  significant  differences  (F(  1 ,92)  =  2.82,  p  <  .09). 


identification  of  the  equipment  is  for  documentation  only  and  does  not  imply  any 
endorsement. 
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L  Static  graphics  with  descnpbcns 

2.  Dynamic  graphics  with  descriptions 

3.  Static  graphics  without  descriptions 

4.  Dynamic  graphics  without  descriptions 

5.  Descriptions  only 

6.  No  preview  instruction  (control) 


Figure  1.  Mean  pretest  scores  for  all  groups. 


The  computer  measured  and  recorded  the  response  time  in  seconds  from  the  moment 
the  jamming  segment  was  presented  in  the  pretest  until  the  student  responded.  Figure  2 
shows  the  mean  response  times  of  all  groups.  (The  mean  response  times  with  standard 
deviations  are  Fsted  in  the  appendix.)  The  response  times  of  the  groups  did  not  differ 
significantly  (F(5,20i)  =  1.00,  p  <  0.4 19).  It  is  interesting  to  note,  however,  that  the 
descriptions-only  and  control  groups  (5,  6),  which  did  not  preview  the  graphics  displays, 
had  the  longest  response  times. 

Figure  3  shows  the  mean  total  time  (in  minutes)  that  the  subjects  spent  actually 
previewing  the  graphics.  (The  mean  total  times  with  standard  deviations  are  listed  in  the 
appendix.)  An  analysis  of  variance  revealed  significant  differences  between  experimental 
groups  (F(4, 1 09)  =  18.09,  p  <  .000).  A  subsequent  test  for  differences  revealed  the  the 
dynamic-graphics-with-descriptions  group  (2)  spent  more  time  viewing  the  graphics  prior 
to  taking  the  pretest  than  did  the  other  groups. 
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Figure  2.  Mean  response  times  for  all  groups. 
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Figure  3.  Mean  total  preview  times  for  all  groups 
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DISCUSSION 


I 


This  experiment  examined  the  overall  effectiveness  of  graphics,  the  value  of 
movement,  and  the  use  of  descriptions  with  the  graphics  in  teaching  students  the  skill  of 
jamming  recognition.  The  results  revealed  that  graphics  were  helpful  and  also  provided 
specific  information  about  the  use  of  movement  and  descriptions. 

The  graphics  groups  (1-4)  scored  higher  than  the  descriptions-only  and  control  groups 
(5,  6)  on  the  pretest  for  the  computer-based  ECM  recognition  training  course.  Therefore, 
the  graphics  provided  these  subjects  with  enough  information  to  recognize  actual 
instances  of  jamming  even  though  they  had  not  seen  actual  jamming  before. 

A  more  detailed  look  at  the  results  showed  that  the  dynamic-graphics-with-descrip- 
tions  group  accounted  for  the  significant  differences,  which  suggests  the  importance  of 
both  movement  and  descriptions.  This  finding  must  be  interpreted  with  caution  because 
of  the  similarity  between  the  dynamic  graphics  and  pretest.  Both  are  moving  pictures. 
Descriptions  appeared  to  add  to  the  value  of  the  dynamic  graphics  although  there  were  no 
main  effects  for  the  descriptions.  The  descriptions  may  have  provided  the  dynamic- 
graphics  group  with  a  more  efficient  way  to  process  and  retain  the  preview  information. 


A 
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Analysis  of  the  time  spent  previewing  the  graphics  provides  another  clue  as  to  the 
reason  for  the  superior  performance  of  the  dynamic-graphics-with-descriptions  group. 
This  group  of  subjects  spent  more  time  previewing  the  graphics  than  did  the  other 
graphics  groups.  Possibly,  this  was  because  the  combination  of  dynamic  graphics  with 
-Ascriptions  gave  them  more  information  to  process.  The  differences  between  the  groups 
in  actual  pretest  response  time  were  not  statistically  significant,  however. 

The  results  of  this  experiment  provide  some  interesting  guidelines  for  the  use  of 
computer  graphics  in  visual  recognition  training.  First,  computer  graphics  appear  to  be  a 
useful  way  of  giving  students  advance  information  about  visual  characteristics  of  complex 
patterns.  The  findings  of  this  study  suggest  that  it  is  important  to  consider  the 
prototypical  features  or  characteristics  of  the  visual  pattern  to  be  learned  in  developing 
the  graphics.  In  our  case,  simplifying  the  visual  pattern  was  important.  However, 
including  the  movement  turned  out  to  be  critical  even  though  it  did  add  some  complexity. 
Finally,  the  findings  suggest  that  the  use  of  descriptions  added  to  the  value  of  the 
dynamic  graphics. 


CONCLUSIONS 

Dynamic  graphics  that  consider  the  characteristics  of  the  visual  patterns  to  be 
learned  appear  to  be  a  useful  way  of  giving  students  advance  information  about  the  visual 
characteristics  of  complex  patterns.  Descriptions  seem  to  add  to  the  value  of  dynamic 
graphics. 


RECOMMENDATION 

Training  developers  should  consider  using  dynamic  computer  graphics  that  focus  on 
the  unique  or  distinguishing  features  of  visual  patterns  to  be  learned  to  give  students 
advance  information  of  visual  characterisitcs  of  complex  patterns. 
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APPENDIX 


MEAN  PRETEST  SCORES,  RESPONSE  TIMES, 

AND  TOTAL  PREVIEW  TIMES  WITH  STANDARD  DEVIATIONS 
FOR  EXPERIMENTAL  AND  CONTROL  GROUPS 


Appendix 


Mean  Pretest  Scores,  Response  Times,  and 
Total  Preview  Times  with  Standard  Deviations 


Group 

N 

Results 

Mean 

SD 

Mean  Pretest  Scores  (%) 

Experimental  groups 

1.  Static  graphics  with  descriptions 

24 

48.41 

21.69 

2.  Dynamic  graphics  with  descriptions 

20 

62,50 

14.46 

3.  Static  graphics  without  descriptions 

25 

46.24 

15.59 

4.  Dynamic  graphics  without  descriptions 

25 

51.16 

15.15 

5.  Descriptions  only 

20 

40.95 

19.58 

Control  groups 

6.  No  preview  instruction 

93 

43.47 

17.89 

Mean  Response  Times  (seconds) 


Experimental  groups 


1. 

Static  graphics  with  descriptions 

24 

40.83 

14.87 

2. 

Dynamic  graphics  with  descriptions 

20 

43.90 

15.03 

3. 

Static  graphics  without  descriptions 

25 

40.68 

15.23 

4. 

Dynamic  graphics  without  descriptions 

25 

41.32 

12.74 

5. 

Descriptions  only 

20 

45.30 

10.40 

Control  groups 

6. 

No  preview  instruction 

93 

45.88 

15.28 

Mean  Total  Preview  Times  (minutes) 

Experimental  groups 

1. 

Static  graphics  with  descriptions 

24 

10.66 

1.46 

2. 

Dynamic  graphics  with  descriptions 

20 

13.10 

1.44 

3. 

Static  graphics  without  descriptions 

25 

9.72 

1.20 

4. 

Dynamic  graphics  without  descriptions 

25 

11.92 

1.28 

5. 

Descriptions  only 

20 

10.45 

1.95 

Control  groups 

6. 

No  preview  instruction 

93 

-- 

-- 
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